Method and system for controlling illuminators

ABSTRACT

A method for controlling illuminators in an eye tracking system and a corresponding system are disclosed. The system includes a plurality of illuminators arranged such that each illuminator of the illuminators is located at a respective fixed position in relation to an eye of a user when using the system. The method comprises illuminating the eye of the user by means of the plurality of illuminators, and receiving an image of the eye of the user from an image sensor, the image resulting from the image sensor detecting light from the plurality of illuminators reflected from the eye of the user and reflected from an optic arrangement located between the plurality of illuminators and the eye of the user. The method further comprises determining a pupil position of a representation in the image of a pupil of the eye, and identifying one or more blobs in the image, wherein a blob is a representation in the image of a reflection from the optic arrangement of light from an illuminator of the plurality of illuminators. On a condition that there is at least one blob of the one or more blobs for which one or more predefined criteria are met, the method comprises identifying an illuminator of the plurality of illuminators associated with the at least one blob of the one or more blobs, and switching off the identified illuminator of the plurality illuminators.

TECHNICAL FIELD

The present disclosure generally relates to the field of eye tracking.In particular, the present disclosure relates to systems and methods foruse in controlling illuminators in an eye tracking system.

BACKGROUND

In eye tracking applications, digital images are retrieved of the eyesof a user and the digital images are analyzed in order to estimate thegaze direction of the user. There are different methods for achievingsuch an estimation. In some methods ambient light is used whenretrieving images of the eyes of the user and in some methods additionallight sources (illuminators) are used to illuminate the eyes forretrieving images of the eyes of the user. Generally, the estimation ofthe gaze is based on identification of the pupils of the eyes of theuser, together with identification of glints (corneal reflections) inthe eyes of the user. In order to identify a pupil of an eye in an imageof the eye, the contrast between a representation of the pupil and arepresentation of an iris of the eye in the image must be sufficient.

One known method of eye tracking includes the use of infrared light andan image sensor. The infrared light is directed towards the pupil of auser and the reflection of the light is captured by an image sensor.Through analysis of the reflection point, the direction of the user'sgaze may be calculated. One such system is described in U.S. Pat. No.7,572,008 (which is hereby incorporated by reference in its entirety).

Portable or wearable eye tracking devices have also been previouslydescribed. One such eye tracking system is described in U.S. Pat. No.9,041,787 (which is hereby incorporated by reference in its entirety). Awearable eye tracking device is described using illuminators and imagesensors for determining gaze direction.

In applications of eye tracking for in portable or wearable eye trackingdevices, such as in virtual reality (VR) devices, where head mounteddevices are used which include an eye tracking system determining an eyedirection and/or gaze direction based on a pupil center and glints fromilluminators illuminating a user's eyes, problems can arise for examplefor a user who is wearing a pair of spectacles under the VR glasses. Forexample, one or more reflections from an optic arrangement of the pairof spectacles caused by the one or more of the illuminators may resultin a situation where the pupil cannot be accurately identified or whereno or too few glints can be identified for eye tracking. In suchsituations it will be difficult or impossible to determine eye directionand/or gaze direction and or eye direction or at least not withdesirable reliability.

It would be desirable to provide an eye tracking technology to accountfor such situations where reflections from glasses worn under VR glassesreduce the accuracy of eye tracking or makes it difficult or impossibleto determine eye direction and/or gaze direction for eye tracking.

SUMMARY

An object of the present disclosure is to address at least one of theissues with known systems and methods.

According to a first aspect, there is provided a method of controllingilluminators in an eye tracking system. The system includes a pluralityof illuminators arranged such that each illuminator of the illuminatorsis located at a respective fixed position in relation to an eye of auser when using the system.

In the method the eye of the user is illuminated by means of theplurality of illuminators and an image of the eye of the user isreceived from an image sensor. The image is the result of the imagesensor detecting light from the plurality of illuminators reflected fromthe eye of the user. In addition to the reflected light from the eye ofthe user, reflected light from an optic arrangement located between theplurality of illuminators and the eye of the user is also detected bythe image sensor.

A pupil position of a representation in the image of a pupil of the eyeis then determined and one or more blobs are identified in the image. Ablob is a representation in the image of a reflection from the opticarrangement of light from an illuminator of the plurality ofilluminators.

On a condition that there is at least one blob of the one or more blobsfor which one or more predefined criteria are met, an illuminator of theplurality of illuminators associated with the at least one blob of theone or more blobs is identified and the identified illuminator of theplurality illuminators is switched off.

As indicated above, some factors that may potentially cause difficultyof determination of gaze direction remain in known method and systemsbased on identification of the pupils of the eyes of the user togetherwith identification of glints (corneal reflections). For example, in asituation where representations of reflections (called blobs herein)from an optic arrangement located between the plurality of illuminatorsand the eye of the user appear in the image in addition therepresentation of the reflections from the eye of the user, such blobsmay affect the possibility to identify the glints and/or therepresentation of the pupil in the image such that it will be difficultor impossible to determine eye direction and/or gaze direction or atleast not with desirable accuracy. Identifying an illuminator thatcauses a blob which meets a one or more criteria, and switching of theidentified illuminator enables cancelling of blobs in the image whichaffect the possibility to identify the representation of the pupil inthe image and/or the glints.

The system may be a wearable system (head mounted system), such as avirtual reality (VR) system where goggles are used arranged on the headof the user, such that illuminators arranged in the head mounted systemwill move together with the head of the user and hence the illuminatorswill be fixed in relation to the eyes of the user when the system isused by the user. It is to be noted that the illuminators are fixed inrelation to the eyes of the user provided that the VR device does notmove in relation to the head during use.

The image sensor can be any suitable type of imaging sensor.

The eye may for example be illuminated by a plurality of illuminatorswhich may for example be light sources such as light emitting diodes andthe light emitted may be visible light or infrared light.

The optic arrangement located between the plurality of illuminators andthe eye of the user when the user is using the eye tracking system maybe any optic arrangement that converges or diverges light, such as alens/glass of a pair of spectacles worn by the user in addition to andseparate to the eye tracking system.

The determination of the pupil position may be done by means of anysuitable method for determination of pupil position.

It is to be noted that illuminators are generally only emitting lightduring exposure of an image. At other times all illuminators would notemit light, e.g. in order to preserve energy which would be particularlyrelevant for a battery driven system. Hence, all illuminators will emitlight only a portion of the time. Indicating that an illuminator isswitched off is intended to mean that the illuminator will not emitlight even when an image is being captured.

In order to maintain the brightness after switching off one or moreilluminators, the luminance of one or more of the illuminators notswitched off the camera's imaging parameters may be controlled tocompensate for the reduced luminance due to the switching off of one ormore illuminators.

In some embodiments, a respective position of the one or more blobs inthe image, a respective distance from the respective position of the oneor more blob to the pupil position, and a respective size of the one ormore blobs are determined. Furthermore, the one or more predefinedcriteria are a predefined distance criterion and/or a predefined sizecriterion. The determined distances and/or sizes are then compared tothe predefined distance criterion and/or the predetermined sizecriterion, respectively. On a condition that the determined distance ofat least one blob of the one or more blobs meets the predefined distancecriterion and/or the determined size of the at least one blob of the oneor more blobs meets the predefined size criterion, an illuminator of theplurality of illuminators associated with the at least one blob of theone or more blobs is identified, and the identified illuminator of theplurality illuminators is switched off.

Identifying an illuminator that causes a blob which meets a distancecriterion and/or a size criterion and switching of the identifiedilluminator enables cancelling of blobs which affect the possibility toidentify glints and/or a representation of the pupil in an image.

The distance criterion may for example be that the determined distanceis below a predefined distance. For example, the predefined distance maybe determined as a distance from the pupil center below which a blob mayaffect the possibility to identify the representation of the pupil inand/or the glints the image and above which a blob may not affect thepossibility to identify the representation of the pupil in the imageand/or the glints.

The size criterion may for example be that the determined size is abovea predefined size. For example, the predefined size may be determined asa size above which a blob may affect the possibility to identify therepresentation of the pupil in the image and/or the glints and belowwhich a blob may not affect the possibility to identify therepresentation of the pupil in the image and/or the glints.

In some embodiments, the pupil position is determined as the position ofthe center of the representation of the pupil in the image.

In some embodiments the position of the blob is determined as theposition the center of mass of the blob. Generally, the shape of theblob is not determined. However, by determining the position of the blobby means of its center of mass will at least to some extent encompassthe shape in relation to the extent to which the blob will affect thepossibility to identify the representation of the pupil in the imageand/or the glints.

In some embodiments, the identification of an illuminator of theplurality of illuminators is based on first dividing the image into twoor more regions. A region of the two or more regions in which the atleast one blob of the one or more blobs is located is identified and asubset of illuminators of the plurality of illuminators, wherein thesubset of illuminator is associated with the identified region.

An illuminator of the identified subset of illuminators is then switchedoff and a further image of the eye is received from the image sensor.The further image is the result of the image sensor detecting light fromthe plurality of illuminators, except the illuminator being switchedoff, reflected from the eye of the user and reflected from the opticarrangement located between the plurality of illuminators and the eye ofthe user.

If the at least one blob of the one or more blobs is still in thefurther image, the illuminator switched off did not cause the at leastone blob of the one or more blobs. Hence, the steps of switching off anilluminator and receiving a further image is repeated for a differentilluminator of the subset of illuminators until the at least one blob ofthe one or more blobs is not in an image resulting from the image sensordetecting light from the plurality of illuminators, except theilluminator currently being switched off, reflected from the eye of theuser and reflected from the optic arrangement located between theplurality of illuminators and the eye of the user.

By dividing the image into at least two regions, the number ofilluminators that need to be switched off to determine if thatilluminator is causing a blob that meets the one or more predefinedcriteria, such as the predefined distance criterion and/or thepredefined size criterion, is reduced. This enables a reduction of theprocessing time to identify the illuminator associated with the blobthat meets the predetermined distance criterion and/or the predeterminedsize criterion.

A subset of illuminators being associated with a region of the image maybe identified based on the positions of each illuminator of the subsetof illuminators. For example, if the image is divided into two regionsby a horizontal line through the center of the image this willcorrespond to a horizontal line through the center of the image sensor.The subset of illuminators including all illuminators arranged above thehorizontal line through the center of the image sensor is associatedwith the region of the image above the horizontal line through thecenter of the image (corresponding to the horizontal line through thecenter of the image sensor).

Division of the image in more than two regions, e.g. four regions usingone horizontal line through the center of the image and one verticalline through the center of the image will reduce the number ofilluminators that will be switched off to determine if that illuminatoris causing a blob which meets the predetermined distance criterionand/or the predetermined size criterion. However, depending on how thesubset of illuminators associated with a region is identified, it is notnecessarily an illuminator of that identified subset of illuminator thatis causing a blob in that region which meets the one or more predefinedcriteria, such as the predefined distance criterion and/or thepredefined size criterion. Hence, there may be a tradeoff betweenprocessing time and the number of times the illuminator that should beswitched off is identified when selecting the number of regions.

In some embodiments, the one or more blobs are further identified in aregion of interest (ROI) of the image, the region of interest being aportion of the image including the representation in the image of thepupil.

The region of interest is defined such that blobs outside the region ofinterest does not affect the possibility to identify the glints and/orthe representation of the pupil in the image, whereas blobs inside theregion of interest may affect the possibility to identify the glintsand/or the representation of the pupil in the image. The region ofinterest may have any suitable shape, such as rectangle, square, circle,ellipse, etc.

Any blobs outside the region of interest are optionally not identifiedor at least not processed further. By limiting the one or more blobsidentified to any blobs in the region of interest, a reduction of theamount of processing required is enabled. At the same time, notprocessing blobs outside the region of interest will not, or at leastonly to a low degree, affect the possibility to identify therepresentation of the pupil and/or the glints in the image depending onthe definition of the region of interest.

In some embodiments, data are recorded on a condition that a blob anilluminator is switched off for a pupil position. For example, data thatmay be recorded is the pupil position, an identification of theilluminator of the plurality illuminators associated with the at leastone blob of the one or more blobs, and, an association between the pupilposition and the identification of the illuminator.

The data recorded in relation to a pupil position can be used forcontrol of illuminators in relation to cancelling blobs in imagescaptured for other pupil positions or images for the same pupil positioncaptured at a later instance in time.

In some embodiments, further data are recorded on a condition that anilluminator is switched off for a pupil position. For example, furtherdata that may be recorded are the position of the at least one blob ofthe one or more blobs and the size of the at least one blob of the oneor more blobs.

In some embodiments where the pupil position, an identification of theilluminator of the plurality illuminators associated with the at leastone blob of the one or more blobs, and an association between the pupilposition and the identification of the illuminator, and possibly theposition and the size of the at least one blob of the one or more blobshave been recorded, the recorded data may be used for further imagesreceived after the pupil has moved from the pupil position in relationto which the data was recorded and then moved back to the pupil positionin relation to which the data was recorded.

In such embodiments, the eye of the user is illuminated by means of theplurality of illuminators and a further image of the eye is receivedfrom the image sensor, the further image resulting from the image sensordetecting light from the plurality of illuminators reflected from theeye of the user and reflected from the optic arrangement located betweenthe plurality of illuminators and the eye of the user. A further pupilposition of a representation in the further image of a pupil of the eyeis determined and for a case where it is determined that the furtherpupil position is equal to the pupil position as recorded, the recordeddata are used to determine an illuminator to switch off. Morespecifically, the recorded data include an association between the pupilposition and the identification of the illuminator associated with theat least one blob of the one or more blobs, i.e. a blob that meets thepredefined distance criterion and/or the predefined size criterion inrelation to the pupil position.

Hence, the recorded data may be used to identify the illuminator to beswitched off without the need for any image processing in relation toidentifying blobs, position and size of blobs and distance to a pupilposition in an image.

In some embodiments where the pupil position, an identification of theilluminator of the plurality illuminators associated with the at leastone blob of the one or more blobs, an association between the pupilposition and the identification of the illuminator, and the position andthe size of the at least one blob of the one or more blobs have beenrecorded, the recorded data may be used for further images receivedafter the pupil has moved from a pupil position in relation to which anilluminator is switched off and data was recorded.

After the pupil has moved, the eye of the user is illuminated by meansof the plurality of illuminators, except the illuminator as identifiedby the identification as recorded. A further image of the eye isreceived from the image sensor, the further image resulting from theimage sensor detecting light from the plurality of illuminators, exceptthe illuminator as identified by the identification as recorded,reflected from the eye of the user and reflected from the opticarrangement located between the plurality of illuminators and the eye ofthe user.

A further pupil position of a representation in further the image of apupil of the eye is determined, and since the pupil has moved, it isdetermining that the further pupil position is different from the pupilposition as recorded.

The recorded data are used in determining a further distance from theposition of the at least one blob as recorded, i.e. a blob that meetsthe predefined distance criterion and optionally the predefined sizecriterion in relation to the pupil position before the pupil had moved,to the further pupil position. On a condition that the determinedfurther distance meets the predefined distance criterion maintaining theilluminator identified by the identification as recorded switched off.

In embodiments where data are recorded such as the pupil position, anidentification of the illuminator of the plurality illuminatorsassociated with the at least one blob of the one or more blobs, anassociation between the pupil position and the identification of theilluminator, and the position and the size of the at least one blob ofthe one or more blobs have been recorded, the recording may be performedcontinuously for each new pupil position for which no data have beenrecorded. For pupil positions for which data have been recorded, therecorded data may be used to switch of an illuminator identified by therecorded data, i.e. switching of an illuminator in order to enableremoval of a blob in the image may be done instantly as soon as aparticular pupil position has been determined.

Hence, the recorded data may be used to identify that a blob for whichthe one or more predefined criteria, such as a predefined distancecriterion and/or a predefined size criterion, are met for a first pupilposition would be a blob for which the one or more predefined criteriaare met also for a second pupil position without the need for any imageprocessing in relation to identifying blobs, position and size of blobsand distance to a pupil position in an image in relation to the secondpupil position. The illuminator causing the blob for which the one ormore predefined criteria are met for the first pupil position can remainswitched off also directly for capturing images for the second pupilposition without the need for first capturing an image for the secondpupil position with the illuminator causing the blob for which the oneor more predefined criteria are met for the first pupil positionswitched on.

In some embodiments, a size threshold for identification of a blob isintroduced for separating identification of a blob from identificationof a glint in the image. For example, a blob can be defined as arepresentation of a reflection in the image having a size larger than ablob threshold size. Any representation of a reflection in the imagelarger than the threshold size is identified as a blob. Representationof a reflection in the image smaller than the threshold size is notidentified as a blob. Such a reflection in the image smaller than thethreshold size may be identified as a glint.

As a blob (representations of reflections from an optic arrangement,such as a lens/glass of a pair of spectacles worn by the user), isgenerally larger in size than a glint (representation of a cornealreflection), a blob may be defined as a continuous representation of abright spot in the image which is larger than a predefined number ofpixels, where the predetermined number of pixels is preferably set suchthat it is larger than the number of pixel of glints and smaller thanthe number of pixels of blobs.

According to a second aspect, there is provided an eye tracking systemcomprising a plurality of illuminators for illuminating an eye of auser, the plurality of illuminators being arranged at fixed positions inrelation to an eye of a user when using the device, and a receiver forreceiving an image of the eye of the user from an image sensor, theimage resulting from the image sensor detecting light from the pluralityof illuminators reflected from the eye of the user and reflected from anoptic arrangement located between the plurality of illuminators and theeye of the user.

The system further comprises processing circuitry. The processingcircuitry may include a processor (e.g., general purpose processor,specialty purpose processor, microprocessor, programmable logic device,etc.) and/or memory (e.g., computer memory such as random access memory,read only memory, etc.). The processing circuitry is arranged todetermine a pupil position of a representation in the image of a pupilof the eye, identifying one or more blob in the image, wherein a blob isa representation in the image of a reflection from the optic arrangementof light from an illuminator of the plurality of illuminators.

The processing circuitry is further arranged to, on a condition that atleast one blob of the one or more blobs meets one or more predefinedcriteria, identify an illuminator of the plurality of illuminatorsassociated with the at least one blob of the one or more blobs, andswitch off the identified illuminator of the plurality illuminators.

In some embodiments, the processing circuitry is further arranged todetermine a respective position of the one or more blobs in the image, arespective distance from the respective position of the one or more blobto the pupil position, and a respective size of the one or more blobs.the processing circuitry is further arranged to, on a condition that thedetermined distance of at least one blob of the one or more blobs meetsa predefined distance criterion and/or the determined size of the atleast one blob of the one or more blobs meets a predefined sizecriterion, identify an illuminator of the plurality of illuminatorsassociated with the at least one blob of the one or more blobs, andswitch off the identified illuminator of the plurality illuminators.

Embodiments of the eye tracking system according to the second aspectmay for example include features corresponding to the features of any ofthe embodiments of the method according to the first aspect.

According to a third aspect, an eye tracking system comprising circuitryconfigured to perform any one of the method of the first aspect and theembodiments of the first aspect.

Embodiments of the eye tracking system according to the second aspectmay for example include features corresponding to the features of any ofthe embodiments of the method according to the first aspect.

According to a fourth aspect, there is provided one or morecomputer-readable storage media storing computer-executable instructionsthat, when executed by an eye tracking system, cause the eye trackingsystem to perform a method according to the first aspect.

Embodiments of the one or more computer-readable storage media accordingto the fourth aspect may for example include features corresponding tothe features of any of the embodiments of the method according to thefirst aspect.

The one or more computer-readable media may for example be one or morenon-transitory computer-readable media.

It is noted that embodiments of the invention relate to all possiblecombinations of features recited in the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplifying embodiments will be described below with reference to theaccompanying drawings:

FIG. 1 shows a schematic view of an eye tacking system, in whichembodiments may be implemented;

FIG. 2 shows an example image of an eye;

FIG. 3a shows a view of selected parts of a head mounted device;

FIG. 3b shows a side view of selected parts of a head mounted device;

FIG. 3c shows an exploded view of selected parts of a head mounteddevice;

FIG. 4 shows a schematic view of an eye tracking system in relation toembodiments;

FIG. 5 shows a schematic view of an eye tracking system in relation toembodiments;

FIG. 6 shows a schematic view of an eye tracking system in relation toembodiments;

FIGS. 7-11 are flow charts of methods according to embodiments.

All the figures are schematic and generally only show parts which arenecessary in order to elucidate the respective embodiments, whereasother parts may be omitted or merely suggested.

DETAILED DESCRIPTION

FIG. 1 shows a simplified view of an eye tacking system 100 (which mayalso be referred to as a gaze tracking system) in a head mounted devicein the form of a virtual reality (VR) device or VR glasses. The system100 comprises illuminators 110-119 for illuminating the eyes of a user,and a light sensor 120 for capturing images of the eyes of the user. Theilluminators 110-119 may for example be light emitting diodes emittinglight in the infrared frequency band, or in the near infrared frequencyband. The light sensor 120 may for example be an image sensor of anytype, such as a complementary metal oxide semiconductor (CMOS) imagesensor or a charged coupled device (CCD) image sensor. The image sensormay consist of an integrated circuit containing an array of pixelsensors, each pixel containing a photodetector and an active amplifier.The image sensor is capable of converting light into digital signals. Inreality, as an example, it could be

-   -   Infrared image sensor or IR image sensor    -   RGB sensor    -   RGBW sensor    -   RGB or RGBW sensor with IR filter

The eye tracking system 100 also comprises circuitry 125, for exampleincluding a receiver 126 and processing circuitry 127, for receiving andprocessing the images captured by the light sensor 120. The circuitry125 may for example be connected to the light sensor 120 and theilluminators 110-119 via a wired or a wireless connection and beco-located with the light sensor 120 and the illuminators 110-119 orlocated at a distance, e.g. in a different device. In another example,the circuitry 125 may be provided in one or more stacked layers belowthe light sensitive surface of the light sensor 120.

It is to be noted that the location of the image sensor 120 in FIG. 1 isonly for illustrative purposes. The location of the sensor for one eyein a VR device is generally away from the line of sight for the user inorder not to obscure a VR display arranged in the VR device for thateye. This is e.g. enabled by means of so called hot mirrors whichreflects a portion of the light and the rest of the light to pass, e.g.infrared light is reflected and visible light is allowed to pass.

FIG. 2 shows an example of an image of an eye 200, captured by the lightsensor 120 of FIG. 1. The circuitry 125 may for example employ imageprocessing (such as digital image processing) for extracting features inthe image. The circuitry 125 may for example employ pupil center corneareflection (PCCR) eye tracking to determine where the eye 200 islooking. In PCCR eye tracking, the position of the center of the pupil210 and the position of the center of a glint 220 at the eye 200 areestimated in the circuitry 125. The glint 220 is caused by reflection oflight from one of the illuminators 110-119. The circuitry 125 calculateswhere the user's eye is in space using the glint 220 and where theuser's eye 200 is pointing using the pupil 210. Since there is typicallyan offset between the optical center of the eye 200 and the fovea, thecircuitry 125 performs calibration of the fovea offset to be able todetermine where the user is looking. The gaze directions obtained fromthe left eye and from the right eye may then be combined to form acombined estimated gaze direction (or viewing direction).

In the eye tracking system described with reference to FIG. 1, theilluminators 110-119 are arranged in an eye tracking module along theperiphery of a circle. This arrangement serves only as an example. Itwill be appreciated that more or less any number of illuminators andlight sensors may be employed for eye tracking, and that suchilluminators and light sensors may be distributed in different waysrelative to displays watched by the user. It will be appreciated thatthe eye tracking scheme described in the present disclosure may forexample be employed for wearable eye tracking (such as in virtualreality (VR) glasses).

FIGS. 3a-c each show a separate view of selected parts of a head mounteddevice in the form of a virtual reality (VR) device (VR glasses) 300including an eye tracking system in which embodiments may beimplemented.

FIG. 3a shows a view of selected parts of a head mounted device in theform of the VR glasses 300 including an eye tracking system in whichembodiments may be implemented. In addition to the VR glasses 300, eyes302 and a head 304 of a user are shown. The VR portion of the VR glasses300 shown comprises two VR displays 305 and two VR lenses 330, one VRdisplay 305 and one VR lens 330 for each eye 302. The VR displays 305are positioned in front of the eyes 302 and the VR lenses 330 arepositioned between the eyes 302 and the VR displays 305. In alternativeto two VR displays 305, two regions of a single VR display may be used.The eye tracking portion of the VR glasses 300 comprises two hot mirrors335 and two cameras 320. In order to capture images of the eyes 302 foruse in eye tracking, the hot mirrors 335 are arranged between the VRdisplays 305 and the VR lenses 330. Furthermore, illuminators (notshown) are arranged on or in the VR glasses 300 such that illuminationrays are directed towards the eyes 302. Reflections from the eyes 302 ofthe illumination rays towards the hot mirrors 335 will reflect towardsthe cameras 320 in which the illumination rays are detected to producean image of the eye. For example, the hot mirrors 335 may be of a typesuch that they will reflect light in the infrared frequency band but betransparent for light in the visible frequency band. The illuminators(not show) used would then produce illumination rays in the infraredfrequency band and the cameras 320 will include image sensors able todetect light in the infrared frequency band.

FIG. 3b shows a side view of selected parts of the VR glasses 300.Illumination rays from the illuminators (not shown) towards the eye 302will reflect back and pass through the VR lens 330 towards the hotmirror 335 and reflect towards the camera 320 in which the illuminationrays are detected to produce an image of the eye.

FIG. 3c shows an exploded view of selected parts of the VR glasses 300.Selected parts for one eye are shown including an illuminator cover 324,illuminators in the form of light emitting diodes (LEDs) 310-319, thecamera 320 including an image sensor, the VR lens 330, a lens cup orlens tube 326, the hot mirror 335, the VR display 305 and an electronicsboard 328. FIG. 3c shows an example arrangement of the illuminators inthe form of LEDs 310-319, where the LEDs 310-319 are arranged along theperiphery of the VR lens 330 to produce a pattern when illuminating theeye 302. The illumination rays from the LEDs 310-319 reflected from theeye and the hot mirror 335 is detected in the camera 320 to produce animage of the eye.

Head mounted devices, such as in VR glasses, can be enhanced byincluding wearable eye tracking using illuminators and one or more lightsensors arranged in the head mounted device for determining eyedirection and/or gaze direction based on estimation of a position of acenter of the pupil and a position of the center of one or more glintsat the eye from the illuminators. A problem that can arise in suchdevices is that when the user wears spectacles (glasses) under the VRglasses, light from the illuminators can be reflected by the glasses ofthe spectacles onto the image sensor. Areas of in an image of the eyeused for eye tracking corresponding to such reflections are called blobsherein. Such blobs may affect the possibility to identify arepresentation of the pupil in the image and/or glints such that it willbe difficult or impossible to determine eye direction and/or gazedirection or at least not with desirable accuracy.

However, as each blob generally is caused by one illuminator, thisilluminator may be identified and switched off. Hence, a new image maybe captured by the image sensor in which the blob is no longer present.

FIG. 4 shows a schematic view of an eye tracking system 400 in relationto embodiments of which examples are disclosed in the following. The eyetracking system 400 may be a head mounted system such as the systemdisclosed in relation to FIGS. 3a-c . The eye tracking system 400includes a plurality of illuminators 410-419, and an image sensor 420.Each illuminator of the plurality of illuminators is located at arespective fixed position in relation to an eye of a user when thesystem user is using the system, i.e. when the system is mounted on thehead of the user. Specifically, the plurality of illuminators 410-419 isarranged along a periphery of a circle in FIG. 4 which may for examplebe the periphery of one of the VR lenses of a pair of VR glasses asillustrated in FIGS. 3a -c.

For illustrative purposes, an image 440 of an eye of the user is shownon the image sensor 420.

Furthermore, it is to be noted that location of the image sensor 420 inFIG. 4 is only for illustrative purposes. The location of the sensor forone eye in a VR device is generally away from the line of sight for theuser in order not to obscure a VR display arranged in the VR device forthat eye. This may be achieved by means of so called hot mirrors whichreflects a portion of the light and the rest of the light to pass, e.g.reflects infrared light and allows visible light to pass.

For eye tracking, the eye of the user by means of the plurality ofilluminators 410-419. The image 440 is the result of the image sensor420 detecting light from the plurality of illuminators 410-419 reflectedfrom the eye of the user and reflected from a glass (not shown) of apair of spectacles worn by the user. The glass of the pair of spectaclesis located between the plurality of illuminators 410-419 and the eye ofthe user. The image 440 of the eye of the user is then received inprocessing circuitry (not shown) from the image sensor 420.

The processing circuitry then processes the image 440 and determines apupil position of a representation in the image of a pupil 450 of theeye. The pupil position may be determined according to any knowprocedure for determining pupil position.

The processing circuitry then further identifies one or more blobs460-463 in the image 440. A blob 460 is a representation in the image440 of light from an illuminator of the plurality of illuminators410-419 reflected from the glass of the pair of spectacles and detectedby the image sensor 420.

As a blob (representations of reflections from an optic arrangement,such as a lens/glass of a pair of spectacles worn by the user), isgenerally larger in size than a glint (representation of a cornealreflection), a blob may be defined as a continuous representation of abright spot in the image which is larger than a predefined number ofpixels, where the predetermined number of pixels is preferably set suchthat it is larger than the number of pixel of glints and smaller thanthe number of pixels of blobs.

The processing circuitry then checks if there are at least one blob 460of the one or more blobs 460-463 that meets one or more predefinedcriteria. On a condition that there is such a blob 460 for which the oneor more predetermined criteria are met, the processing circuitry goes onto identify which illuminator of the plurality of illuminators 410-419that is causing the blob 460 and switches off the identifiedilluminator.

Hence, when a new image is captured by the image sensor 420 resultingfrom the image sensor 420 detecting light from all illuminators of theplurality of illuminators 410-419 except the identified illuminator, thenew image will not include the blob 460 for which the one or morepredefined criteria are met.

In embodiments, the one or more predefined criteria comprises apredefined distance criterion and a predefined size criterion. Thepredefined distance criterion may for example be a criterion that thedistance from the blob to the pupil position is less than a predefineddistance. The predefined size criterion may for example be a criterionthat the size of the blob is greater than a predefined size. In suchembodiments, the processing circuitry further determines a respectiveposition of the one or more blobs 460-463 in the image, and a respectivedistance from the respective position of the one or more blobs 460-463to the pupil position. The processing circuitry further determines arespective size of the one or more blobs 460-463.

The pupil position may for example be defined as the center of therepresentation of the pupil 450 in the image 440. The position of a blobmay be defined as the center of mass of the blob. The size of a blob maybe defined as a number of pixels and may for example be calculated usingMueen's Algorithm for Similarity Search (MASS) for the blob.

The processing circuitry then checks if there is at least one blob 460for which the predefined distance criterion is met and/or the predefinedsize criterion is met. On a condition that there is such a blob 460, theprocessing circuitry goes on to identify which illuminator of theplurality of illuminators 410-419 that is causing the at least one blob460 of the one or more blobs 460-463 and switches off the identifiedilluminator.

Hence, when a new image is captured by the image sensor 420 resultingfrom the image sensor 420 detecting light from all illuminators of theplurality of illuminators 410-419 except the identified illuminator, thenew image will not include the blob 460 for which the predefineddistance criterion is met and/or the predefined size criterion is met.

In some cases, two or more blobs are identified for which the predefineddistance criterion is met and/or the predefined size criterion are met,for example that these two or more blobs both have a distance to thepupil position that is less than a predefined distance according to thedistance criterion and/or both blobs have a size that is greater than apredefined size according to the size criterion. For such cases, theilluminators of the plurality of illuminators 410-419 causing the two ormore blobs can be identified and switched off. In alternative, a blob ofthe two or more blobs which is closest in distance to the pupil centerof the two or more blobs can be identified and an illuminator of theplurality of illuminators 410-419 can be identified and switched off andnone of the other illuminators of the plurality of illuminators 410-419are switched off. If more than one blob of the two or more blobs areclosest in distance to the pupil center, an illuminator of the pluralityof illuminators 410-419 causing the blob of the more than one blobs thatis largest in size is identified and switched off and none of the otherilluminators of the plurality of illuminators 410-419 are switched off.

In another alternative for cases where two or more blobs are identifiedfor which the predefined distance criterion is met and/or the predefinedsize criterion is met, a blob of the two or more blobs which has alargest size can be identified and an illuminator of the plurality ofilluminators 410-419 can be identified and switched off and none of theother illuminators of the plurality of illuminators 410-419 are switchedoff. If more than one illuminator of the two or more illuminators arelargest in size, an illuminator of the plurality of illuminators 410-419causing the blob of the more than one blobs that is closest in distanceto the pupil center is identified and switched off and none of the otherilluminators of the plurality of illuminators 410-419 are switched off.

As a further alternative, a combination of the distance criterion andthe size criterion may be used, optionally using weights between the twocriteria.

In embodiments, the illuminator of the plurality of illuminators 410-419that causes a blob 460 in a first image 440 for which the one or morepredefined criteria, such as the predefined distance criterion and/orthe predefined size criterion, are met can be identified by means ofdividing the image into two or more regions. For example, four regionsmay be defined such as the four regions indicated in FIG. 4 by means ofthe horizontal and vertical dashed lines.

A region of the four regions in which the blob 460 for which the one ormore criteria is met is identified. The blob 460 for which the one ormore criteria is met is located in the lower left quarter of the image440. A subset of illuminators associated with the identified region isthen identified. For example, illuminators 417-419 arranged in a regionof FIG. 4 corresponding to the region in which the blob 460 is locatedcan be identified as the subset of illuminators. This would be based onthe assumption that illuminators which in some terms are closest to thelocation of a blob are most likely to have caused the blob when VR lenscenter coincide with the image center.

Once the illuminators 417-419 of the subset of illuminators have beenidentified, a first illuminator 417 of the subset of illuminators isswitched off. A second image of the eye is then captured by and receivedfrom the image sensor 420. The second image is the result of the imagesensor 420 detecting light from the plurality of illuminators, exceptthe first illuminator 417 being switched off, reflected from the eye ofthe user and reflected from the optic arrangement located between theplurality of illuminators 417-419 and the eye of the user.

It is then checked if the blob 460 for which the one or more predefinedcriteria were met in relation to the first image is still in the secondimage. If it is not, the first illuminator 417 that was switched of inrelation to the second image is identified as the illuminator thatcaused the blob 460 for which the one or more predefined criteria weremet in relation to the first image. If the blob 460 for which the one ormore predefined criteria were met in relation to the first image isstill in the second image, the first illuminator 417 is switched on anda second illuminator 418, is switched off. A third image of the eye isthen captured by and received from the image sensor 420. The third imageis the result of the image sensor 420 detecting light from the pluralityof illuminators, except the second illuminator 418 being switched off,reflected from the eye of the user and reflected from the opticarrangement located between the plurality of illuminators 417-419 andthe eye of the user.

It is then checked if the blob 460 for which the one or more predefinedcriteria were met in relation to the first image is still in the thirdimage. If it is not, the second illuminator 418 that was switched of inrelation to the third image is identified as the illuminator that causedthe blob 460 for which the one or more predefined criteria were met inrelation to the first image. If the blob 460 for which the one or morepredefined criteria were met in relation to the first image is still inthe third image, the second illuminator 418 is switched on and a thirdilluminator 419, is switched off. A fourth image of the eye is thencaptured by and received from the image sensor 420. The fourth image isthe result of the image sensor 420 detecting light from the plurality ofilluminators, except the third illuminator 419 being switched off,reflected from the eye of the user and reflected from the opticarrangement optic arrangement located between the plurality ofilluminators 417-419 and the eye of the user.

It is then checked if the blob 460 for which the one or more predefinedcriteria were met in relation to the first image is still in the secondimage. If it is not, the third illuminator 419 that was switched of inrelation to the fourth image is identified as the illuminator thatcaused the blob 460 for which the one or more predefined criteria weremet in relation to the first image. If the blob 460 for which the one ormore predefined criteria were met in relation to the first image isstill in the second image, the third illuminator 419 is switched on. Inthis case no one of the illuminators in the subset of illuminators417-419 associated with the region in which the blob 460 is located, itmay be decided that no illuminator should be switched off or search forthe illuminator causing the blob 460 is continued in other regions,preferably starting with illuminators in the region closest to the blob,which in FIG. 4 would be the upper left quarter.

Generally, after an illuminator has been switched off, it may benecessary to maintain an overall brightness of an image for glint and/orpupil detection. For example, the image needs to be bright enough forrobust glint and/or pupil detection.

For example, one or more of the following may be used in order tomaintain overall brightness or at least to some extent compensate for anilluminator being switched off:

-   -   Increase the luminance of other illuminators than the        illuminator being switched off. This could be done by        controlling the current output for one or more of the other        illuminators    -   Increase the camera aperture    -   Increase ISO of the camera    -   Decrease the shutter speed of the camera

In summary, to maintain the brightness after switching off one or moreilluminators, the luminance of one or more of the illuminators notswitched off the camera's imaging parameters control to compensate forthe reduced luminance due to the switching off of one or moreilluminators.

FIG. 5 shows a schematic view of an eye tracking system in relation toembodiments in which a region of interest (ROI) 570 of the image isdefined.

For illustrative purposes, an image 540 of an eye of the user is shownon the image sensor 520.

Furthermore, it is to be noted that location of the image sensor 520 inFIG. 5 is only for illustrative purposes. The location of the sensor forone eye in a VR device is generally away from the line of sight for theuser in order not to obscure a VR display arranged in the VR device forthat eye. This may be achieved by means of so called hot mirrors whichreflects a portion of the light and the rest of the light to pass, e.g.reflects infrared light and allows visible light to pass.

The region of interest 570 is a portion of the image including therepresentation in the image of the pupil 550. The region of interest 570is preferably defined such that blobs 561-563 outside the region ofinterest 570 do not, or only in a low degree, affect the possibility toidentify the representation of the pupil 550 and/or the glints in theimage 540, whereas a blob 560 inside the region of interest 570 mayaffect the possibility to identify the representation of the pupil 550and/or the glints in the image 540. The region of interest 570 isillustrated in FIG. 5 in dashed lines as a square but may have any othersuitable shape, such as rectangle, circle, ellipse, etc. Any blobs561-563 outside the region of interest 570 are not identified or atleast not processed further. By limiting the blobs identified andfurther processed to any blobs 560 in the region of interest 570, theamount of processing required can be reduced. At the same time, notprocessing blobs 561-563 outside the region 570 of interest may not, oronly to a low degree, affect the possibility to identify therepresentation of the pupil 550 and/or the glints in the image 540depending on the definition of the region of interest 570.

FIG. 6 shows a schematic view of an eye tracking system in relation toembodiments of which examples are disclosed in the following.

For illustrative purposes, an image 640 of an eye of the user is shownon the image sensor 620.

Furthermore, it is to be noted that location of the image sensor 620 inFIG. 6 is only for illustrative purposes. The location of the sensor forone eye in a VR device is generally away from the line of sight for theuser in order not to obscure a VR display arranged in the VR device forthat eye. This may be achieved by means of so called hot mirrors whichreflects a portion of the light and the rest of the light to pass, e.g.reflects infrared light and allows visible light to pass.

After identifying one or more illuminators causing one or more blobs forwhich the one or more predefined criteria, such as the predefineddistance criterion and/or the predefined size criterion, are met for apupil position, data can be recorded in relation to different pupilpositions.

In FIG. 6, images 640 for three different pupil positions 650-652 areshown superposed for illustrative purposes. The pupil is shown when ithas moved to a first pupil position 650, to a second pupil position 651and to a third pupil position 652 in the three superposed images 640.

On a condition that an illuminator is switched off for a pupil position,data that may be recorded in relation to that pupil position. If forexample, a blob close to the representation of the pupil in in a firstimage associated with the first pupil position 650 is identified as ablob 660 for which the one or more predefined criteria, such as thepredefined distance criterion and/or the predefined size criterion, aremet, then an illuminator of the plurality of illuminators causing theblob 660 should be switched off for the first pupil position 650. Theilluminator causing the blob 660 may for example be identified by meansof one of the embodiments disclosed herein. After identification of theilluminator causing the blob 660 in the first image relating to thefirst pupil position 650, data can be stored in relation to the firstpupil position 650.

Examples of data recorded are:

-   -   the first pupil position 650    -   an identification of the illuminator causing the blob 660 for        which the one or more criteria are met    -   an association between the first pupil position 650 and the        identification of the illuminator are recorded    -   the position of the blob 660 for which the one or more criteria        are met    -   the size of the blob 660 for which the one or more criteria are        met

In embodiments where data have been recorded, the recorded data may beused for a further image captured and received after the pupil has movedfrom the first pupil position 650 in relation to which the data wererecorded to a second pupil position 651 and then back to the first pupilposition 650. In such embodiments, the eye of the user is illuminated bymeans of the plurality of illuminators 610-619 and a further image ofthe eye is received from the image sensor 620, the further imageresulting from the image sensor 620 detecting light from the pluralityof illuminators 610-619 reflected from the eye of the user and reflectedfrom a glass (not shown) of a pair of spectacles located between theplurality of illuminators 610-619 and the eye of the user. A pupilposition of a representation in the further image of the pupil of theeye is determined and it is determined that the pupil position in thefurther image is equal to the first pupil position 650. The recordeddata associated with the first pupil position 650 are then used todetermine which illuminator to switch off. More specifically, therecorded data include the association between the first pupil position650 and the identification of the illuminator associated with the blob660 for which the one or more criteria are met. Hence, the recorded datamay be used to identify the illuminator to be switched off without theneed for any image processing in relation to identifying blobs, positionand size of blobs and distance to a pupil position in an image from eachidentified blob.

In embodiments where data have been recorded in relation to a firstpupil position, such as the example above where a blob close to therepresentation of the pupil in in a first image associated with thefirst pupil position 650 was identified as a blob 660 for which thepredefined distance criterion was met and optionally the predefined sizecriterion was met and data were recorded associated with the first pupilposition 650. The recorded data associated with the first pupil position650 may be used in relation to capturing a second image after the pupilhas moved from the first pupil position 650 in relation to which thedata were recorded to a second pupil position 651.

After the pupil has moved to the second pupil position 651, theilluminator identified by the identification as recorded in relation tothe first pupil position 650 is not immediately switched on even thoughthe second pupil position 651 is different from the first pupil position650. Instead, the eye of the user is illuminated by means of theplurality of illuminators 610-619, except the illuminator as identifiedby the identification as recorded associated with the first pupilposition 650. A second image of the eye is captured by and received fromthe image sensor 620, the second image resulting from the image sensordetecting light from the plurality of illuminators 610-619, except theilluminator as identified by the identification as recorded for thefirst pupil position 650, reflected from the eye of the user andreflected from the optic arrangement located between the plurality ofilluminators 610-619 and the eye of the user.

The second pupil position 651 of the representation in the second imageof the pupil of the eye is determined, and since the pupil has moved, itis determined that the second pupil position 651 is different from thefirst pupil position 650 as recorded.

The recorded data are used in determining a further distance from theposition of the blob 660 as recorded associated with the first pupilposition 650, i.e. the blob 660 for which the predefined distancecriterion was met and optionally the predefined size criterion was metin relation to the first pupil position 650, to the second pupilposition 651. On a condition that the determined further distance meetsthe predefined distance criterion, the illuminator identified by theidentification as recorded in relation to the first pupil position ismaintained switched off. On a condition that the determined furtherdistance does not meet the predefined distance criterion, theilluminator identified by the identification as recorded in relation tothe first pupil position is switched on.

In embodiments where data are recorded, the recording may be performedcontinuously for each new pupil position, e.g. for the first pupilposition 650, the second pupil position 651 and the third pupil position652, for which no data have been previously recorded. For pupilpositions for which data have been recorded, the recorded data may forexample be used to switch off an illuminator identified by the recordeddata, i.e. removal of a blob in the image may be done instantly as soonas a particular pupil position has been determined.

FIG. 7 shows a method according to an embodiment of controllingilluminators in an eye tracking system, wherein the system includes aplurality of illuminators arranged such that each illuminator of theilluminators is located at a respective fixed position in relation to aneye of a user when using the system. The method comprises illuminating710 the eye of the user by means of the plurality of illuminators. Themethod further comprises receiving 720 an image of the eye of the userfrom an image sensor, the image resulting from the image sensordetecting light from the plurality of illuminators reflected from theeye of the user and reflected from an optic arrangement located betweenthe plurality of illuminators and the eye of the user. The methodfurther comprises determining 730 a pupil position of a representationin the image of a pupil of the eye, and identifying 740 one or moreblobs in the image, wherein a blob is a representation in the image of areflection from the optic arrangement of light from an illuminator ofthe plurality of illuminators. On a condition that there is at least oneblob of the one or more blobs for which one or more predefined criteriaare met, the method further comprises identifying 750 an illuminator ofthe plurality of illuminators associated with the at least one blob ofthe one or more blobs and switching off 760 the identified illuminatorof the plurality of illuminators.

FIG. 8 shows a method according to another embodiment of controllingilluminators in an eye tracking system, wherein the system includes aplurality of illuminators arranged such that each illuminator of theilluminators is located at a respective fixed position in relation to aneye of a user when using the system. The method according to anembodiment. The method comprises illuminating 810 the eye of the user bymeans of the plurality of illuminators. The method further comprisesreceiving 820 an image of the eye of the user from an image sensor, theimage resulting from the image sensor detecting light from the pluralityof illuminators reflected from the eye of the user and reflected from anoptic arrangement located between the plurality of illuminators and theeye of the user. The method further comprises determining 830 a pupilposition of a representation in the image of a pupil of the eye, andidentifying 840 one or more blobs in the image, wherein a blob is arepresentation in the image of a reflection from the optic arrangementof light from an illuminator of the plurality of illuminators. Themethod further comprises determining 850 a respective position of theone or more blobs in the image, determining 860 a respective distancefrom the respective position of the one or more blob to the pupilposition, and determining 870 a respective size of the one or moreblobs. On a condition that there is at least one blob of the one or moreblobs for which the determined distance of the at least one blob of theone or more blobs meets a predefined distance criterion and/or thedetermined size of the at least one blob of the one or more blobs meetsa predefined size criterion, the method further comprises identifying880 an illuminator of the plurality of illuminators associated with theat least one blob of the one or more blobs and switching off 890 theidentified illuminator of the plurality of illuminators.

FIG. 9 shows a method in relation to embodiments in which identificationof an illuminator of a plurality of illuminator is causing a blob forwhich one or more criteria are met. The method comprises dividing 910the image into two or more regions, identifying 920 a region of the twoor more regions in which the blob for which the one or more criteria aremet is located, and identifying 930 a subset of illuminators of theplurality of illuminators, the subset of illuminator being associatedwith the identified region. The method further comprises switching off940 an illuminator of the identified subset of illuminators, receiving950 a further image of the eye from the image sensor, the further imageresulting from the image sensor detecting light from the plurality ofilluminators, except the illuminator being switched off, reflected fromthe eye of the user and reflected from the optic arrangement locatedbetween the plurality of illuminators and the eye of the user. On acondition 970 that the at least one blob of the one or more blobs is inthe further image switching on 980 the illuminator that was switched ofin relation to the further image, and repeating, for a differentilluminator of the subset of illuminators, the steps of switching off940 an illuminator and receiving 950 a further image. On a condition 970that the at least one blob of the one or more blobs is not in thefurther image, identifying 990 the illuminator being switched off inrelation to the further image as the illuminator causing the blob forwhich one or more criteria are met.

FIG. 10 shows a method in relation to embodiments in which data arerecorded for a pupil position in relation to a blob for which one ormore predefined criteria are met and an illuminator causing the blob isswitched off.

The method comprises recording 1010:

-   -   the pupil position    -   an identification of the illuminator of the plurality        illuminators associated with the blob for which the one or more        criteria are met    -   an association between the pupil position and the identification        of the illuminator    -   the position of the blob for which the one or more criteria are        met    -   the size of the blob for which the one or more criteria are met

The method further comprises illuminating 1020 the eye of the user bymeans of the plurality of illuminators, and receiving 1030 a furtherimage of the eye from the image sensor, the further image resulting fromthe image sensor detecting light from the plurality of illuminatorsreflected from the eye of the user and reflected from the spectacleglass located between the plurality of illuminators and the eye of theuser. The method further comprises determining 1040 a further pupilposition of a representation in further the image of a pupil of the eye,determining 1050 that the further pupil position is equal to the pupilposition as recorded, and switching off 1060 the illuminator asidentified by the identification as recorded and association to thepupil position as recorded.

FIG. 11 shows a method in relation to embodiments in which data arerecorded for a pupil position in relation to a blob for which apredefined distance criterion is met and/or a predefined size criterionis met and an illuminator causing the blob is switched off.

The method comprises recording 1110:

-   -   the pupil position    -   an identification of the illuminator of the plurality        illuminators associated with the blob for which the predefined        distance criterion is met and/or the predefined size criterion        is met    -   an association between the pupil position and the identification        of the illuminator    -   the position of the blob for which the predefined distance        criterion is met and/or the predefined size criterion is met    -   the size of the blob for which the predefined distance criterion        and/or the predefined size criterion are met

The method further comprises illuminating 1120 the eye of the user bymeans of the plurality of illuminators, except the illuminator asidentified by the identification as recorded, and receiving 1130 afurther image of the eye from the image sensor, the further imageresulting from the image sensor detecting light from the plurality ofilluminators, except the illuminator as identified by the identificationas recorded, reflected from the eye of the user and reflected from theoptic arrangement located between the plurality of illuminators and theeye of the user. The method further comprises determining 1140 a furtherpupil position of a representation in further the image of a pupil ofthe eye, determining 1150 that the further pupil position is differentfrom the pupil position as recorded, and determining 1160 a furtherdistance from the position of the at least one blob as recorded to thefurther pupil position. On a condition that the determined furtherdistance meets the predefined distance criterion the method comprisesmaintaining 1170 the illuminator identified by the identification asrecorded switched off.

A person skilled in the art realizes that the present invention is by nomeans limited to the preferred embodiments described above. On thecontrary, many modifications and variations are possible within thescope of the appended claims. For example, the person skilled in the artrealizes that the methods described herein may be performed by othereye/gaze tracking systems than the example eye/gaze tracking system 100shown in FIG. 1, for example any eye/gaze tracking system in which thelocation of illuminators is fixed in relation to the eyes of a user whenthe system is used. Furthermore, the embodiments in relation to FIGS.4-9 have been described for one eye. However, a person skilled in theart realizes that the methods may be performed for two eyes also wherethe result for both eyes is taken into account in each step.

Additionally, variations to the disclosed embodiments can be understoodand effected by those skilled in the art in practicing the claimedinvention, from a study of the drawings, the disclosure, and theappended claims. In the claims, the word “comprising” does not excludeother elements or steps, and the indefinite article “a” or “an” does notexclude a plurality. The division of tasks between functional unitsreferred to in the present disclosure does not necessarily correspond tothe division into physical units; to the contrary, one physicalcomponent may have multiple functionalities, and one task may be carriedout in a distributed fashion, by several physical components incooperation. A computer program may be stored/distributed on a suitablenon-transitory medium, such as an optical storage medium or asolid-state medium supplied together with or as part of other hardware,but may also be distributed in other forms, such as via the Internet orother wired or wireless telecommunication systems. The mere fact thatcertain measures/features are recited in mutually different dependentclaims does not indicate that a combination of these measures/featurescannot be used to advantage. Method steps need not necessarily beperformed in the order in which they appear in the claims or in theembodiments described herein, unless it is explicitly described that acertain order is required. Any reference signs in the claims should notbe construed as limiting the scope.

The invention claimed is:
 1. A method of controlling illuminators in aneye tracking system, the method comprising: illuminating an eye of auser via a plurality of illuminators, wherein the plurality ofilluminators are arranged such that each illuminator of the illuminatorsis located at a respective fixed position in relation to the eye of theuser when using the eye tracking system; receiving an image of the eyeof the user from an image sensor, the image resulting from the imagesensor detecting light from the plurality of illuminators reflected fromthe eye of the user and reflected from an optic arrangement locatedbetween the plurality of illuminators and the eye of the user;determining a pupil position of a representation in the image of a pupilof the eye; identifying one or more blobs in the image, wherein a blobis a representation in the image of a reflection from the opticarrangement of light from an illuminator of the plurality ofilluminators; and on a condition that there is at least one blob of theone or more blobs for which one or more predefined criteria are met:identifying an illuminator of the plurality of illuminators associatedwith the at least one blob of the one or more blobs; and switching offthe identified illuminator of the plurality illuminators.
 2. The methodof claim 1, further comprising: determining a respective position of theone or more blobs in the image; determining a respective distance fromthe respective position of the one or more blob to the pupil position;and determining a respective size of the one or more blobs, wherein theone or more predefined criteria comprises a predefined distancecriterion and/or a predefined size criterion and wherein on a conditionthat the determined distance of at least one blob of the one or moreblobs meets a predefined distance criterion and/or the determined sizeof the at least one blob of the one or more blobs meets a predefinedsize criterion: identifying an illuminator of the plurality ofilluminators associated with the at least one blob of the one or moreblobs; and switching off the identified illuminator of the pluralityilluminators.
 3. The method of claim 2, wherein the pupil position isdetermined as the position of the center of the representation of thepupil in the image and the position of a blob is determined as theposition of the center of mass of the blob.
 4. The method of claim 3,wherein the distance criterion is met on a condition that the determineddistance is below a predefined distance.
 5. The method of claim 3,wherein the size criterion is met on a condition that the determinedsize is above a predefined size.
 6. The method of claim 2, furthercomprising: recording, on a condition that an illuminator is switchedoff, the pupil position, an identification of the illuminator of theplurality illuminators associated with the at least one blob of the oneor more blobs, an association between the pupil position and theidentification of the illuminator, the position of the at least one blobof the one or more blobs, the size of the at least one blob of the oneor more blobs.
 7. The method of claim 6, further comprising: on acondition that an illuminator is switched off: illuminating the eye ofthe user by means of the plurality of illuminators, except theilluminator as identified by the identification as recorded; receiving afurther image of the eye from the image sensor, the further imageresulting from the image sensor detecting light from the plurality ofilluminators, except the illuminator as identified by the identificationas recorded, reflected from the eye of the user and reflected from theoptic arrangement located between the plurality of illuminators and theeye of the user; determining a further pupil position of arepresentation in further the image of a pupil of the eye; determiningthat the further pupil position is different from the pupil position asrecorded; determining a further distance from the position of the atleast one blob as recorded to the further pupil position; and on acondition that the determined further distance meets the predefineddistance criterion maintaining the illuminator identified by theidentification as recorded switched off.
 8. The method of claim 1,wherein identifying an illuminator of the plurality of illuminatorscomprises: dividing the image into two or more regions, identifying aregion of the two or more regions in which the at least one blob of theone or more blobs is located; identifying a subset of illuminators ofthe plurality of illuminators, the subset of illuminator beingassociated with the identified region; switching off an illuminator ofthe identified subset of illuminators; receiving a further image of theeye from the image sensor, the further image resulting from the imagesensor detecting light from the plurality of illuminators, except theilluminator being switched off, reflected from the eye of the user andreflected from the optic arrangement located between the plurality ofilluminators and the eye of the user; on a condition that the at leastone blob of the one or more blobs is in the further image, repeating,for a different illuminator of the subset of illuminators, the steps ofswitching off an illuminator and receiving a further image until the atleast one blob of the one or more blobs is not in the further image. 9.The method of claim 1, wherein the one or more blobs are furtheridentified in a region of interest of the image, the region of interestbeing a portion of the image including the representation in the imageof the pupil.
 10. The method of claim 1, further comprising: on acondition that an illuminator is switched off, recording: the pupilposition, an identification of the illuminator of the pluralityilluminators associated with the at least one blob of the one or moreblobs, and an association between the pupil position and theidentification of the illuminator.
 11. The method of claim 10, furthercomprising: illuminating the eye of the user using the plurality ofilluminators; receiving a further image of the eye from the imagesensor, the further image resulting from the image sensor detectinglight from the plurality of illuminators reflected from the eye of theuser and reflected from the spectacle glass located between theplurality of illuminators and the eye of the user; determining a furtherpupil position of a representation in further the image of a pupil ofthe eye; determining that the further pupil position is equal to thepupil position as recorded; and switching off the illuminator asidentified by the identification as recorded and association to thepupil position as recorded.
 12. The method of claim 1, wherein a blob isa representation of a reflection in the image having a size larger thana blob threshold size.
 13. An eye tracking system comprising: aplurality of illuminators configured to illuminate an eye of a user, theplurality of illuminators being arranged at fixed positions in relationto an eye of a user when using the device; a receiver configured toreceive an image of the eye of the user from an image sensor, the imageresulting from the image sensor detecting light from the plurality ofilluminators reflected from the eye of the user and reflected from anoptic arrangement located between the plurality of illuminators and theeye of the user; processing circuitry configured to: determine a pupilposition of a representation in the image of a pupil of the eye,identifying one or more blob in the image, wherein a blob is arepresentation in the image of a reflection from the optic arrangementof light from an illuminator of the plurality of illuminators; and on acondition that at least one blob of the one or more blobs meets one ormore predefined criteria: identify an illuminator of the plurality ofilluminators associated with the at least one blob of the one or moreblobs; and switch off the identified illuminator of the pluralityilluminators.
 14. The system of claim 13, wherein the one or morepredefined criteria comprises a predefined distance criterion and apredefined size criterion, and wherein the processing circuitry isfurther configured to: determine a respective position of the one ormore blobs in the image; determine a respective distance from therespective position of the one or more blob to the pupil position;determine a respective size of the one or more blobs; and on a conditionthat the determined distance of at least one blob of the one or moreblobs meets a predefined distance criterion and/or the determined sizeof the at least one blob of the one or more blobs meets a predefinedsize criterion: identify an illuminator of the plurality of illuminatorsassociated with the at least one blob of the one or more blobs; andswitch off the identified illuminator of the plurality illuminators. 15.A eye tracking system comprising circuitry configured to implement amethod comprising: illuminating an eye of a user via a plurality ofilluminators, wherein the plurality of illuminators are arranged suchthat each illuminator of the illuminators is located at a respectivefixed position in relation to the eye of the user when using the eyetracking system; receiving an image of the eye of the user from an imagesensor, the image resulting from the image sensor detecting light fromthe plurality of illuminators reflected from the eye of the user andreflected from an optic arrangement located between the plurality ofilluminators and the eye of the user; determining a pupil position of arepresentation in the image of a pupil of the eye; identifying one ormore blobs in the image, wherein a blob is a representation in the imageof a reflection from the optic arrangement of light from an illuminatorof the plurality of illuminators; and on a condition that there is atleast one blob of the one or more blobs for which one or more predefinedcriteria are met: identifying an illuminator of the plurality ofilluminators associated with the at least one blob of the one or moreblobs; and switching off the identified illuminator of the pluralityilluminators.
 16. The eye tracking system of claim 15, the methodfurther comprising: determining a respective position of the one or moreblobs in the image; determining a respective distance from therespective position of the one or more blob to the pupil position; anddetermining a respective size of the one or more blobs, wherein the oneor more predefined criteria comprises a predefined distance criterionand/or a predefined size criterion and wherein on a condition that thedetermined distance of at least one blob of the one or more blobs meetsa predefined distance criterion and/or the determined size of the atleast one blob of the one or more blobs meets a predefined sizecriterion: identifying an illuminator of the plurality of illuminatorsassociated with the at least one blob of the one or more blobs; andswitching off the identified illuminator of the plurality illuminators.17. The eye tracking system of claim 16, wherein the pupil position isdetermined as the position of the center of the representation of thepupil in the image and the position of a blob is determined as theposition of the center of mass of the blob.
 18. One or morecomputer-readable storage media storing computer-executable instructionsthat, when executed by a processor, cause the processor to perform amethod comprising: illuminating an eye of a user via a plurality ofilluminators, wherein the plurality of illuminators are arranged suchthat each illuminator of the illuminators is located at a respectivefixed position in relation to the eye of the user when using the eyetracking system; receiving an image of the eye of the user from an imagesensor, the image resulting from the image sensor detecting light fromthe plurality of illuminators reflected from the eye of the user andreflected from an optic arrangement located between the plurality ofilluminators and the eye of the user; determining a pupil position of arepresentation in the image of a pupil of the eye; identifying one ormore blobs in the image, wherein a blob is a representation in the imageof a reflection from the optic arrangement of light from an illuminatorof the plurality of illuminators; and on a condition that there is atleast one blob of the one or more blobs for which one or more predefinedcriteria are met: identifying an illuminator of the plurality ofilluminators associated with the at least one blob of the one or moreblobs; and switching off the identified illuminator of the pluralityilluminators.